Lyrics segmentation via bimodal text–audio representation

Abstract Song lyrics contain repeated patterns that have been proven to facilitate automated segmentation, with the final goal of detecting building blocks (e.g., chorus, verse) a song text. Our contribution in this article is twofold. First, we introduce convolutional neural network (CNN)-based model learns segment based on their repetitive text structure. We experiment novel features reveal different kinds repetitions lyrics, for instance phonetical and syntactical properties. Second, using corpus where synchronized audio song, show modalities capture complementary structure combining both beneficial segmentation performance. For purely text-based dataset 103k achieve an F-score 67.4%, improving state art (59.2% F-score). On text–audio 4.8k songs, additional improve performance 75.3% F-score, significantly outperforming approaches.